Comprehensive Code Review Implementation Plan
For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
Status: Active
Last Updated: 2025-11-02
Related Docs: Design Document, Python Standards, Architecture Patterns
Code Location: engineering/ (entire Python codebase)
Goal: Review and fix all 134 Python files to meet engineering standards through automated compliance pass, deep manual review of critical files, and comprehensive sweep of remaining files.
Architecture: Three-phase approach - Phase 1 uses automated tools (Black, Ruff, mypy) for quick compliance wins, Phase 2 performs deep 7-point manual inspection of 15 critical files, Phase 3 systematically reviews remaining 119 files with 5-point streamlined checklist.
Tech Stack: Black (formatting), Ruff (linting + isort), mypy (type checking), pytest (testing), Git (version control)
Pre-Implementation: Setup Tasks
Task 0: Create Progress Tracker
Files:
- Create: engineering/code-review-progress.md
Step 1: Create progress tracking file
cat > engineering/code-review-progress.md << 'EOF'
# Code Review Progress
**Started**: 2025-11-02
**Status**: In Progress
**Current Phase**: 1 of 3
## Phase Completion
- [ ] Phase 1: Automated Compliance (0/134 files)
- [ ] Phase 2: Critical Path Review (0/15 files)
- [ ] Phase 3: Comprehensive Sweep (0/119 files)
## Session Log
### Session 1 (2025-11-02)
- Phase: 1 (Automated)
- Files completed: 0
- Token usage: 0/200000
EOF
Step 2: Commit progress tracker
git add engineering/code-review-progress.md
git commit -m "docs: create code review progress tracker
Track progress across 3 phases:
- Phase 1: Automated (134 files)
- Phase 2: Critical (15 files)
- Phase 3: Comprehensive (119 files)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Task 1: Tighten Tool Configuration
Files:
- Modify: pyproject.toml:44-50 (mypy section)
- Modify: pyproject.toml:22-42 (ruff section)
Step 1: Update mypy configuration for strict type checking
Edit pyproject.toml mypy section:
[tool.mypy]
python_version = "3.9"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true # ← Changed from false (STRICT)
warn_unused_ignores = true # ← New
strict_optional = true # ← New
ignore_missing_imports = true
check_untyped_defs = true
Step 2: Add Ruff docstring checking
Add to pyproject.toml ruff section:
[tool.ruff]
line-length = 100
target-version = "py39"
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # pyflakes
"I", # isort
"C", # flake8-comprehensions
"B", # flake8-bugbear
"UP", # pyupgrade
"D", # pydocstyle (docstrings) ← New
]
ignore = [
"E501", # line too long (handled by black)
"B008", # do not perform function calls in argument defaults
"C901", # too complex (we'll handle complexity case by case)
]
[tool.ruff.pydocstyle]
convention = "google" # Enforce Google-style docstrings
Step 3: Test configuration changes
cd backend/epgoat
mypy --version
ruff --version
black --version
Expected: All tools installed and versions displayed
Step 4: Commit configuration changes
git add pyproject.toml
git commit -m "chore: tighten Python tool configuration for strict standards
- Enable strict type checking in mypy (disallow_untyped_defs)
- Add docstring linting with Ruff (pydocstyle)
- Enforce Google-style docstrings convention
Prepares for comprehensive code review.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Phase 1: Automated Compliance Pass
Task 2: Batch 1 - Core Modules (20 files)
Files:
- Modify: backend/epgoat/core/*.py (~20 files)
Step 1: Run isort to fix import order
cd backend/epgoat
ruff check --select I --fix core/
Expected: Import order fixed automatically
Step 2: Run Black to format code
black core/
Expected: "X files reformatted" or "All done!"
Step 3: Run Ruff auto-fix for safe violations
ruff check --fix core/
Expected: Safe violations fixed automatically
Step 4: Run mypy to identify type hint gaps
mypy core/ > /tmp/mypy-core-violations.txt
cat /tmp/mypy-core-violations.txt
Expected: List of type errors to fix manually
Step 5: Fix type hint violations manually
For each file with type errors:
1. Read the file
2. Add missing type hints based on mypy output
3. Run mypy core/<filename>.py to verify
Example fix:
# Before
def get_channel(channel_id):
return channels.get(channel_id)
# After
def get_channel(channel_id: str) -> Optional[Channel]:
return channels.get(channel_id)
Step 6: Validate all checks pass
make ci
Expected: All tests pass, no linting errors, no type errors
Step 7: Commit Batch 1
git add backend/epgoat/core/
git commit -m "refactor: apply automated standards to core/ modules (Batch 1/6)
- Fix import order (isort via Ruff)
- Apply Black formatting
- Add missing type hints
- Fix Ruff violations
Files: ~20 core modules
Phase 1 progress: 20/134 (15%)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 8: Update progress tracker
Mark Batch 1 complete in engineering/code-review-progress.md
Task 3: Batch 2 - Services Layer (25 files)
Files:
- Modify: backend/epgoat/services/*.py (~25 files)
Step 1: Run automated tools sequence
cd backend/epgoat
ruff check --select I --fix backend/epgoat/services/
black backend/epgoat/services/
ruff check --fix backend/epgoat/services/
Step 2: Generate type hint gap report
mypy backend/epgoat/services/ > /tmp/mypy-services-violations.txt
cat /tmp/mypy-services-violations.txt
Step 3: Fix type hint violations
Work through mypy violations file by file, adding type hints.
Step 4: Validate
make ci
Step 5: Commit Batch 2
git add backend/epgoat/services/
git commit -m "refactor: apply automated standards to backend/epgoat/services/ layer (Batch 2/6)
- Fix import order, formatting, linting
- Add missing type hints to all service files
Files: ~25 service modules
Phase 1 progress: 45/134 (34%)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 6: Update progress tracker
Task 4: Batch 3 - Pipeline Components (15 files)
Files:
- Modify: backend/epgoat/pipeline/*.py (~15 files)
Repeat same sequence as Batch 2:
- Run automated tools (isort, Black, Ruff)
- Generate mypy report
- Fix type hints
- Validate with
make ci - Commit with progress message (60/134, 45%)
- Update tracker
Task 5: Batch 4 - Database Layer (18 files)
Files:
- Modify: backend/epgoat/infrastructure/database/*.py (~18 files)
Repeat same sequence:
- Automated tools
- mypy report
- Fix type hints (special attention to repository patterns)
- Validate
- Commit (78/134, 58%)
- Update tracker
Task 6: Batch 5 - Utilities & Helpers (30 files)
Files:
- Modify: backend/epgoat/utils/*.py + backend/epgoat/helpers/*.py (~30 files)
Repeat same sequence:
- Automated tools
- mypy report
- Fix type hints
- Validate
- Commit (108/134, 81%)
- Update tracker
Task 7: Batch 6 - Tests (26 files)
Files:
- Modify: backend/epgoat/tests/*.py (~26 files)
Note: Tests have relaxed type hint requirements but still need formatting
Step 1: Run automated tools
cd backend/epgoat
black tests/
ruff check --fix tests/
Step 2: Run mypy (informational only)
mypy tests/ || true # Don't fail on test type errors
Step 3: Validate tests still pass
make test
Step 4: Commit Batch 6
git add backend/epgoat/tests/
git commit -m "refactor: apply formatting standards to tests/ (Batch 6/6)
- Apply Black formatting
- Fix Ruff violations
- Tests remain passing
Phase 1 COMPLETE: 134/134 files (100%)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 5: Update progress tracker - Phase 1 complete
Task 8: Phase 1 Validation
Step 1: Run full CI suite
make ci
Expected: All tests pass, no linting errors, no type errors
Step 2: Generate Phase 1 summary report
cat >> engineering/code-review-progress.md << 'EOF'
## Phase 1 Summary
**Status**: Complete ✅
**Files processed**: 134/134
**Batches**: 6
**Violations fixed**:
- Type hints added: [count from mypy reports]
- Formatting issues: [count from Black]
- Linting issues: [count from Ruff]
- Import order: [count from isort]
**All files now pass:**
- Black formatting ✅
- Ruff linting ✅
- mypy type checking ✅
- All tests passing ✅
EOF
Step 3: Commit Phase 1 completion
git add engineering/code-review-progress.md
git commit -m "docs: complete Phase 1 automated compliance
All 134 Python files now meet basic standards:
- 100% type hint coverage
- Black formatted
- Ruff clean
- Tests passing
Next: Phase 2 (Critical Path Review)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 4: Push to remote
git push
Phase 2: Critical Path Deep Review
Task 9: Review api_enrichment.py (Critical File 1/15)
Files:
- Read: backend/epgoat/services/api_enrichment.py
- Modify: backend/epgoat/services/api_enrichment.py (if violations found)
Step 1: Read the file completely
# Use Read tool to load the entire file
Step 2: 7-Point Inspection
1. Architecture Review: - Check SOLID principles compliance - Verify separation of concerns - Check dependency injection - Identify design patterns used
2. Type Safety:
- Verify 100% type hint coverage (already done in Phase 1)
- Check for Any types - should use specific types
- Verify proper use of generics
- Check Optional handling
3. Documentation: - Verify all public functions have Google-style docstrings - Check for Args, Returns, Raises, Examples sections - Verify complex algorithms are explained - Check module docstring exists
4. Error Handling:
- No bare except: clauses
- Specific exception types
- Proper logging
- Graceful degradation
5. Testing:
- Check test file exists (tests/backend/epgoat/services/test_api_enrichment.py)
- Verify >80% coverage
- Check edge cases tested
6. Performance: - No N+1 queries - Appropriate caching - Efficient algorithms
7. Security: - Input validation - No SQL injection risks - Secrets in environment variables
Step 3: Document violations found
Create a violation list for this file:
### api_enrichment.py Violations
**Architecture:**
- [ ] Function `process_channels` >50 lines (Line 145-210) - extract helper methods
- [ ] Direct database access instead of repository pattern (Line 89)
**Documentation:**
- [ ] Missing docstring on `_internal_helper` (Line 234)
- [ ] Missing Examples section in `enrich_event` docstring
**Error Handling:**
- [ ] Bare except clause (Line 178) - should catch specific exceptions
- [ ] No logging for API failures (Line 156)
**Testing:**
- [ ] No test for error path when API rate limited
- [ ] Missing integration test for full enrichment flow
**Performance:**
- [ ] Repeated API calls in loop (Line 145-160) - should batch
**Security:**
- [ ] No validation of channel.name before regex (Line 92)
Step 4: Fix violations immediately
For each violation: 1. Fix the issue in code 2. Run tests to verify 3. Update violation list with ✅
Example fix:
# Before
try:
result = api_call()
except: # Bare except - violation
return None
# After
try:
result = api_call()
except (APIError, ConnectionError) as e: # Specific exceptions
logger.error(f"API call failed: {e}")
return None
Step 5: Validate fixes
cd backend/epgoat
mypy backend/epgoat/services/api_enrichment.py
ruff check backend/epgoat/services/api_enrichment.py
pytest tests/backend/epgoat/services/test_api_enrichment.py -v
Expected: All checks pass
Step 6: Commit api_enrichment.py fixes
git add backend/epgoat/services/api_enrichment.py tests/backend/epgoat/services/test_api_enrichment.py
git commit -m "refactor(services): improve api_enrichment.py quality (Critical 1/15)
7-Point inspection complete:
- Extract long functions into helpers
- Add missing docstrings
- Fix error handling (specific exceptions)
- Add input validation
- Improve test coverage
Phase 2 progress: 1/15 (7%)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 7: Update progress tracker
Task 10-23: Review Remaining Critical Files (14 files)
Repeat Task 9 pattern for each critical file:
Task 10: regex_matcher.py (Critical 2/15) Task 11: enhanced_league_inference.py (Critical 3/15) Task 12: patterns.py (Critical 4/15) Task 13: league_normalizer.py (Critical 5/15) Task 14: database/repositories/event_repository.py (Critical 6/15) Task 15: database/repositories/match_repository.py (Critical 7/15) Task 16: database/schema_validator.py (Critical 8/15) Task 17: database/d1_client.py (Critical 9/15) Task 18: backend/epgoat/services/thesportsdb_client.py (Critical 10/15) Task 19: api/handlers.py (Critical 11/15) Task 20: middleware/error_handler.py (Critical 12/15) Task 21: epg_generator.py (Critical 13/15) Task 22: pipeline/schedulers.py (Critical 14/15) Task 23: pipeline/xmltv.py (Critical 15/15)
For each file: 1. Read completely 2. 7-point inspection 3. Document violations 4. Fix immediately 5. Validate 6. Commit individually 7. Update tracker
Task 24: Phase 2 Validation
Step 1: Run full test suite
make test-coverage
Expected: >80% coverage on all critical files
Step 2: Generate Phase 2 summary
cat >> engineering/code-review-progress.md << 'EOF'
## Phase 2 Summary
**Status**: Complete ✅
**Files reviewed**: 15/15 critical files
**Violations fixed**: [total count]
**Test coverage**: >80% on all critical paths
**Architecture improvements**: [list major refactors]
**Critical files now meet:**
- SOLID principles ✅
- 100% type safety ✅
- Complete documentation ✅
- Robust error handling ✅
- Comprehensive testing ✅
- Performance optimized ✅
- Security hardened ✅
EOF
Step 3: Commit Phase 2 completion
git add engineering/code-review-progress.md
git commit -m "docs: complete Phase 2 critical path review
15 critical files deeply reviewed and improved:
- Matching pipeline (5 files)
- Data integrity (4 files)
- API integration (3 files)
- Core pipeline (3 files)
All critical systems now meet 7-point quality standards.
Next: Phase 3 (Comprehensive Sweep)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 4: Push to remote
git push
Phase 3: Comprehensive Sweep
Task 25: High Priority Batch 1 - Utilities (30 files)
Files:
- Modify: backend/epgoat/utils/*.py + backend/epgoat/helpers/*.py (~30 files)
5-Point Streamlined Review per file:
Step 1: Review file against checklist
For each file: 1. Docstring Completeness: All public functions have Google-style docstrings? 2. Type Hint Correctness: Types accurate and specific (not just present)? 3. Error Handling: Appropriate exceptions, proper logging? 4. Code Complexity: Functions <50 lines, complexity <10? 5. YAGNI Violations: Dead code, over-engineering, unused imports?
Step 2: Document violations
Create violation list for batch.
Step 3: Fix violations
Process files in groups of 5-10 for efficiency.
Step 4: Validate
make ci
Step 5: Commit batch
git add backend/epgoat/utils/ backend/epgoat/helpers/
git commit -m "refactor: comprehensive review of utilities/helpers (Phase 3 - Batch 1/5)
5-point review complete for 30 files:
- Docstrings added/improved
- Type hints corrected
- Error handling enhanced
- Complexity reduced
- Dead code removed
Phase 3 progress: 30/119 (25%)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 6: Update tracker
Task 26: Medium Priority Batch 2 - Models & Parsers (40 files)
Files:
- Modify: backend/epgoat/domain/*.py + backend/epgoat/parsers/*.py (~40 files)
Repeat 5-point review pattern:
- Review against checklist
- Document violations
- Fix issues
- Validate
- Commit (70/119, 59%)
- Update tracker
Task 27: Medium Priority Batch 3 - Formatters & Validators (40 files)
Files:
- Modify: backend/epgoat/formatters/*.py + backend/epgoat/validators/*.py (~40 files)
Repeat 5-point review pattern:
- Review
- Document
- Fix
- Validate
- Commit (110/119, 92%)
- Update tracker
Task 28: Lower Priority Batch 4 - Scripts & Tools (9 files)
Files:
- Modify: engineering/scripts/*.py + engineering/tools/*.py (~9 files)
Repeat 5-point review pattern:
- Review
- Document
- Fix
- Validate
- Commit (119/119, 100%)
- Update tracker
Task 29: Phase 3 Validation
Step 1: Run complete validation
make ci
python3 scripts/validate_documentation_standards.py # Check docstrings
Step 2: Generate Phase 3 summary
cat >> engineering/code-review-progress.md << 'EOF'
## Phase 3 Summary
**Status**: Complete ✅
**Files reviewed**: 119/119 remaining files
**Batches**: 4
**Violations fixed**: [total count]
**All files now meet:**
- Complete docstrings ✅
- Accurate type hints ✅
- Proper error handling ✅
- Reduced complexity ✅
- No YAGNI violations ✅
EOF
Step 3: Commit Phase 3 completion
git add engineering/code-review-progress.md
git commit -m "docs: complete Phase 3 comprehensive sweep
All 119 remaining files reviewed:
- 30 utilities/helpers
- 40 backend/epgoat/domain/parsers
- 40 formatters/validators
- 9 scripts/tools
100% of codebase now meets engineering standards.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 4: Push to remote
git push
Final Validation & Reporting
Task 30: Generate Final Code Review Report
Files:
- Create: engineering/CODE-REVIEW-REPORT.md
Step 1: Create comprehensive summary report
cat > engineering/CODE-REVIEW-REPORT.md << 'EOF'
# Code Review Report
**Date Completed**: 2025-11-02
**Scope**: 134 Python files in engineering/
**Approach**: Three-phase (Automated → Critical → Comprehensive)
---
## Summary
✅ **100% of codebase reviewed and fixed**
### Phase 1: Automated Compliance
- **Files**: 134/134
- **Violations fixed**: [total]
- Type hints added: [count]
- Formatting issues: [count]
- Linting issues: [count]
### Phase 2: Critical Path Review
- **Files**: 15/15
- **7-point inspection**: PASS
- **Test coverage**: >80%
- **Major refactors**: [list]
### Phase 3: Comprehensive Sweep
- **Files**: 119/119
- **5-point review**: PASS
- **Docstrings added**: [count]
- **Complexity reduced**: [count files]
---
## Standards Compliance
All files now meet:
✅ Python Standards (01-Python-Standards.md)
✅ Architecture Patterns (03-Architecture-Patterns.md)
✅ Testing Standards (04-Testing-Standards.md)
✅ Documentation Standards (05-Documentation-Standards.md)
---
## Tool Compliance
✅ **Black**: 100% formatted
✅ **Ruff**: 0 violations
✅ **mypy**: 0 type errors
✅ **pytest**: All tests passing
---
## Key Improvements
### Architecture
<div class="kanban-column">
- [List major architectural improvements]
</div>
### Type Safety
- Added [count] type hints
- Eliminated all `Any` types
- Proper use of generics throughout
### Documentation
<div class="kanban-column">
- [count]</div>
docstrings added
- All public APIs documented
- Complex algorithms explained
### Error Handling
- Eliminated all bare `except` clauses
- Specific exception types used
- Comprehensive logging added
### Testing
- Coverage improved to >80%
- Edge cases covered
- Integration tests added
### Performance
<div class="kanban-column">
- [List performance improvements]
</div>
### Security
- Input validation added
- SQL injection risks eliminated
- Secrets properly managed
---
## Recommendations
### Immediate
<div class="kanban-column">
- [List any immediate follow-up items]
</div>
### Short-term
<div class="kanban-column">
- [List short-term improvements]
</div>
### Long-term
<div class="kanban-column">
- [List long-term architecture improvements]
</div>
---
## Conclusion
The EPGOAT codebase now meets all engineering standards. All 134 Python files have been reviewed and improved for quality, maintainability, and security.
**Next Steps**: Maintain standards through CI/CD enforcement and code review process.
EOF
Step 2: Commit report
git add engineering/CODE-REVIEW-REPORT.md
git commit -m "docs: add final code review report
Comprehensive review complete:
- 134 Python files reviewed
- 100% standards compliance
- 3 phases executed successfully
All code now meets engineering standards.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Task 31: Update Progress Tracker - COMPLETE
Step 1: Mark all phases complete
cat > engineering/code-review-progress.md << 'EOF'
# Code Review Progress
**Started**: 2025-11-02
**Status**: COMPLETE ✅
**Completed**: 2025-11-02
## Phase Completion
- [x] Phase 1: Automated Compliance (134/134) ✅
- [x] Phase 2: Critical Path Review (15/15) ✅
- [x] Phase 3: Comprehensive Sweep (119/119) ✅
## Summary
**Total files reviewed**: 134
**Total violations fixed**: [count]
**Sessions used**: [count]
**Total tokens**: [count]
All Python code now meets engineering standards.
See: CODE-REVIEW-REPORT.md for detailed report.
EOF
Step 2: Final commit
git add engineering/code-review-progress.md
git commit -m "docs: mark code review complete
🎉 ALL 134 FILES REVIEWED
- Phase 1: Automated compliance ✅
- Phase 2: Critical path review ✅
- Phase 3: Comprehensive sweep ✅
100% standards compliance achieved.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 3: Push final changes
git push
Task 32: CI/CD Integration (Optional but Recommended)
Files:
- Modify: .github/workflows/ci.yml
Step 1: Add strict standards enforcement to CI
Add to .github/workflows/ci.yml:
- name: Check code formatting
run: |
cd backend/epgoat
black --check .
- name: Check linting
run: |
cd backend/epgoat
ruff check .
- name: Check type hints
run: |
cd backend/epgoat
mypy .
Step 2: Commit CI updates
git add .github/workflows/ci.yml
git commit -m "ci: enforce strict code standards in CI/CD
All PRs must now pass:
- Black formatting check
- Ruff linting (including docstrings)
- mypy type checking (strict mode)
Prevents regressions from code review work.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Step 3: Push CI changes
git push
Success Criteria
Phase 1 Complete When:
- ✅ All 134 files pass
make ci - ✅ mypy reports 0 type errors with
disallow_untyped_defs = true - ✅ Black reports 0 formatting issues
- ✅ Ruff reports 0 linting violations
- ✅ Progress tracker updated
Phase 2 Complete When:
- ✅ All 15 critical files pass 7-point inspection
- ✅ Test coverage >80% on critical paths
- ✅ All immediate fixes applied
- ✅ TODO comments added for complex refactors
- ✅ Progress tracker updated
Phase 3 Complete When:
- ✅ All 119 remaining files reviewed
- ✅ All violations fixed or documented
- ✅ Progress tracker shows 100%
- ✅ All tests passing
Overall Success:
- ✅ 100% files meet engineering standards
- ✅ 0 type errors across entire codebase
- ✅ All tests passing
- ✅ Complete docstrings on all public APIs
- ✅ Progress tracker complete
- ✅ Final report published
- ✅ CI/CD enforcing standards
Execution Notes
Token Management
- Estimated total: 600k-850k tokens
- Expected sessions: 6-9 sessions
- Checkpoint frequency: After each batch/file
- Progress tracking: Continuous updates
Session Handoff
When approaching token limit (<20k): 1. Complete current file/batch 2. Commit changes 3. Update progress tracker 4. Push to remote 5. Note next file/batch in progress tracker
Resuming
- Read
engineering/code-review-progress.md - Find last completed item
- Resume at next file/batch
- Continue with same standards